Scalable Parallel Algorithms for Solving Sparse Systems of Linear Equations∗

We have developed a highly parallel sparse Cholesky factorization algorithm that substantially improves the state of the art in parallel direct solution of sparse linear systems—both in terms of scalability and overall performance. It is a well known fact that dense matrix factorization scales well and can be implemented efficiently on parallel computers. However, it had been a challenge to developing efficient and scalable parallel formulations of sparse matrix factorization. Our new parallel sparse factorization algorithm is asymptotically as scalable as the best dense matrix factorization algorithms for a wide class of problems that include all twoand three-dimensional finite element problems. This algorithm incurs less communication overhead than any previously known parallel formulation of sparse matrix factorization. It is equally scalable on parallel architectures based on 2-D mesh, hypercube, fat-tree, and multistage networks. In addition, it is the only known sparse factorization algorithm that can deliver speedups in proportion to an increasing number of processors while requiring almost constant memory per processor. We have successfully implemented this algorithm for Cholesky factorization on nCUBE2 and Cray T3D parallel computers. An implementation of this algorithm on the T3D delivers up to 20 GFlops on 1024 processors for medium-size structural engineering and linear programming problems. To the best of our knowledge, this is the highest performance ever obtained for sparse Cholesky factorization on any supercomputer. Numerical factorization is the most time consuming of the four phases involved in obtaining a direct solution of a sparse system of linear equations. In addition to Cholesky factorization, we present efficient parallel algorithms for two other phases—symbolic factorization and for forward and backward substitution to solve the triangular systems resulting from sparse matrix factorization. These algorithms are designed to work in conjunction with our sparse Cholesky factorization algorithm and incur less communication overhead than parallel sparse Cholesky factorization. Along with some recently developed parallel ordering algorithms, the algorithms presented in this thesis make it possible to develop complete scalable parallel direct solvers for sparse linear systems.